The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Font generation is a difficult and time-consuming task, especially in those languages using ideograms that have complicated structures with a large number of characters, such as Chinese. To solve this problem, few-shot font generation and even one-shot font generation have attracted a lot of attention. However, most existing font generation methods may still suffer from (i) large cross-font gap challenge; (ii) subtle cross-font variation problem; and (iii) incorrect generation of complicated characters. In this paper, we propose a novel one-shot font generation method based on a diffusion model, named Diff-Font, which can be stably trained on large datasets. The proposed model aims to generate the entire font library by giving only one sample as the reference. Specifically, a large stroke-wise dataset is constructed, and a stroke-wise diffusion model is proposed to preserve the structure and the completion of each generated character. To our best knowledge, the proposed Diff-Font is the first work that developed diffusion models to handle the font generation task. The well-trained Diff-Font is not only robust to font gap and font variation, but also achieved promising performance on difficult character generation. Compared to previous font generation methods, our model reaches state-of-the-art performance both qualitatively and quantitatively.

Cross-modality magnetic resonance (MR) image synthesis aims to produce missing modalities from existing ones. Currently, several methods based on deep neural networks have been developed using both source- and target-modalities in a supervised learning manner. However, it remains challenging to obtain a large amount of completely paired multi-modal training data, which inhibits the effectiveness of existing methods. In this paper, we propose a novel Self-supervised Learning-based Multi-scale Transformer Network (SLMT-Net) for cross-modality MR image synthesis, consisting of two stages, \ie, a pre-training stage and a fine-tuning stage. During the pre-training stage, we propose an Edge-preserving Masked AutoEncoder (Edge-MAE), which preserves the contextual and edge information by simultaneously conducting the image reconstruction and the edge generation. Besides, a patch-wise loss is proposed to treat the input patches differently regarding their reconstruction difficulty, by measuring the difference between the reconstructed image and the ground-truth. In this case, our Edge-MAE can fully leverage a large amount of unpaired multi-modal data to learn effective feature representations. During the fine-tuning stage, we present a Multi-scale Transformer U-Net (MT-UNet) to synthesize the target-modality images, in which a Dual-scale Selective Fusion (DSF) module is proposed to fully integrate multi-scale features extracted from the encoder of the pre-trained Edge-MAE. Moreover, we use the pre-trained encoder as a feature consistency module to measure the difference between high-level features of the synthesized image and the ground truth one. Experimental results show the effectiveness of the proposed SLMT-Net, and our model can reliably synthesize high-quality images when the training set is partially unpaired. Our code will be publicly available at https://github.com/lyhkevin/SLMT-Net.

Dynamic networks have been extensively explored as they can considerably improve the model's representation power with acceptable computational cost. The common practice in implementing dynamic networks is to convert given static layers into fully dynamic ones where all parameters are dynamic and vary with the input. Recent studies empirically show the trend that the more dynamic layers contribute to ever-increasing performance. However, such a fully dynamic setting 1) may cause redundant parameters and high deployment costs, limiting the applicability of dynamic networks to a broader range of tasks and models, and more importantly, 2) contradicts the previous discovery in the human brain that \textit{when human brains process an attention-demanding task, only partial neurons in the task-specific areas are activated by the input, while the rest neurons leave in a baseline state.} Critically, there is no effort to understand and resolve the above contradictory finding, leaving the primal question -- to make the computational parameters fully dynamic or not? -- unanswered. The main contributions of our work are challenging the basic commonsense in dynamic networks, and, proposing and validating the \textsc{cherry hypothesis} -- \textit{A fully dynamic network contains a subset of dynamic parameters that when transforming other dynamic parameters into static ones, can maintain or even exceed the performance of the original network.} Technically, we propose a brain-inspired partially dynamic network, namely PAD-Net, to transform the redundant dynamic parameters into static ones. Also, we further design Iterative Mode Partition to partition the dynamic- and static-subnet, which alleviates the redundancy in traditional fully dynamic networks. Our hypothesis and method are comprehensively supported by large-scale experiments with typical advanced dynamic methods.

Adapter Tuning, which freezes the pretrained language models (PLMs) and only fine-tunes a few extra modules, becomes an appealing efficient alternative to the full model fine-tuning. Although computationally efficient, the recent Adapters often increase parameters (e.g. bottleneck dimension) for matching the performance of full model fine-tuning, which we argue goes against their original intention. In this work, we re-examine the parameter-efficiency of Adapters through the lens of network pruning (we name such plug-in concept as \texttt{SparseAdapter}) and find that SparseAdapter can achieve comparable or better performance than standard Adapters when the sparse ratio reaches up to 80\%. Based on our findings, we introduce an easy but effective setting ``\textit{Large-Sparse}'' to improve the model capacity of Adapters under the same parameter budget. Experiments on five competitive Adapters upon three advanced PLMs show that with proper sparse method (e.g. SNIP) and ratio (e.g. 40\%) SparseAdapter can consistently outperform their corresponding counterpart. Encouragingly, with the \textit{Large-Sparse} setting, we can obtain further appealing gains, even outperforming the full fine-tuning by a large margin. Our code will be released at: https://github.com/Shwai-He/SparseAdapter.

现有检测方法通常使用参数化边界框（Bbox）进行建模和检测（水平）对象，并将其他旋转角参数用于旋转对象。我们认为，这种机制在建立有效的旋转检测回归损失方面具有根本的局限性，尤其是对于高精度检测而言，高精度检测（例如0.75）。取而代之的是，我们建议将旋转的对象建模为高斯分布。一个直接的优势是，我们关于两个高斯人之间距离的新回归损失，例如kullback-leibler Divergence（KLD）可以很好地对齐实际检测性能度量标准，这在现有方法中无法很好地解决。此外，两个瓶颈，即边界不连续性和正方形的问题也消失了。我们还提出了一种有效的基于高斯度量的标签分配策略，以进一步提高性能。有趣的是，通过在基于高斯的KLD损失下分析Bbox参数的梯度，我们表明这些参数通过可解释的物理意义进行了动态更新，这有助于解释我们方法的有效性，尤其是对于高精度检测。我们使用量身定制的算法设计将方法从2-D扩展到3-D，以处理标题估计，并在十二个公共数据集（2-D/3-D，空中/文本/脸部图像）上进行了各种基本检测器的实验结果。展示其优越性。

我们描述了JD Explore Academy对WMT 2022共享的一般翻译任务的提交。我们参加了所有高资源曲目和一条中型曲目，包括中文英语，德语英语，捷克语英语，俄语 - 英语和日语英语。我们通过扩大两个主要因素，即语言对和模型大小，即\ textbf {vega-mt}系统来推动以前的工作的极限 - 进行翻译的双向培训。至于语言对，我们将“双向”扩展到“多向”设置，涵盖所有参与语言，以利用跨语言的常识，并将其转移到下游双语任务中。至于型号尺寸，我们将变压器限制到拥有近47亿参数的极大模型，以完全增强我们VEGA-MT的模型容量。此外，我们采用数据增强策略，例如单语数据的循环翻译以及双语和单语数据的双向自我训练，以全面利用双语和单语言数据。为了使我们的Vega-MT适应通用域测试集，设计了概括调整。根据受约束系统的官方自动分数，根据图1所示的sacrebleu，我们在{zh-en（33.5），en-zh（49.7）（49.7），de-en（33.7）上获得了第一名-de（37.8），CS-EN（54.9），En-CS（41.4）和En-Ru（32.7）}，在{ru-en（45.1）和Ja-en（25.6）}和第三名上的第二名和第三名在{en-ja（41.5）}上； W.R.T彗星，我们在{zh-en（45.1），en-zh（61.7），de-en（58.0），en-de（63.2），cs-en（74.7），ru-en（ru-en（ru-en）上，我们获得了第一名64.9），en-ru（69.6）和en-ja（65.1）}，分别在{en-cs（95.3）和ja-en（40.6）}上的第二名。将发布模型，以通过GitHub和Omniforce平台来促进MT社区。

本文试图通过域的适应来建立新兴超模型范式的理论基础，其中首先训练一个非常大的模型，{\ it i.e.}，超级模型（或其他论文中的基础模型），大量数据，然后将其调整到各种特定域。超模型范式有助于减少计算和数据成本和碳排放，这对AI行业至关重要，尤其是中小型企业。我们将超模型范式建模为两个阶段的扩散过程：（1）在训练阶段，模型参数从随机缩写和收敛到稳定分布； （2）在微调阶段，模型参数被运输到另一个稳定分布。两个训练阶段都可以通过Uhlenbeck-ornstein过程进行数学建模，该过程分别收敛到两个Maxwell-Boltzmann分布，每个分布都表征了相应的收敛模型。然后，通过PAC-Bayesian Framework建立了$ \ Mathcal O（1/\ sqrt {n}）$概括。该理论发现，微调阶段的概括误差在域适应中是主要的。此外，我们的理论表明，概括是由一种新的度量确定的，该新度量是根据协方差矩阵和融合局部最小值的转移来表征源域和目标域之间域差异的。

大型语言模型（LLM）已在一系列自然语言理解任务上实现了最先进的表现。但是，这些LLM可能依靠数据集偏差和文物作为预测的快捷方式。这极大地损害了他们的分布（OOD）概括和对抗性鲁棒性。在本文中，我们对最新发展的综述，这些发展解决了LLMS的鲁棒性挑战。我们首先介绍LLM的概念和鲁棒性挑战。然后，我们介绍了在LLM中识别快捷方式学习行为的方法，表征了快捷方式学习的原因以及引入缓解解决方案。最后，我们确定了关键挑战，并将这一研究线的联系引入其他方向。

数十亿人每天都在社交媒体上分享他们的日常生活图像。但是，它们的生物识别信息（例如，指纹）可以很容易地从这些图像中偷走。从社交媒体上泄漏的指纹泄漏的威胁引起了人们对匿名分享图像的强烈渴望，同时保持图像质量，因为指纹充当了终生的个体生物识别密码。为了防止指纹泄漏，通过在图像上添加不可察觉的扰动来作为解决方案出现。但是，现有作品要么在黑盒可传输性方面弱，要么显得不自然。由视觉感知层次结构激励（即，高级感知利用模型共享的语义，这些语义在模型中很好地转移，而低水平的感知提取物则是原始刺激的，并且会引起高视觉敏感性的刺激），我们提出了一个层次的感知噪声，注射框架以解决上述问题。对于黑盒可传递性，我们在指纹方向场上注入保护性噪声，以扰动模型共享的高级语义（即指纹脊）。考虑到视觉自然性，我们通过正规化侧向基因核的响应来抑制低级局部对比度刺激。我们的Fingersafe是第一个在数字（最高94.12％）和现实的场景（Twitter和Facebook，高达68.75％）中提供可行的指纹保护的人。我们的代码可以在https://github.com/nlsde-safety-team/fingersafe上找到。